360-degree virtual-reality system for dynamic events

ABSTRACT

A dynamic event capturing and rendering system collects and aggregates video, audio, positional, and motion data to create a comprehensive user perspective 360-degree rendering of a field of play. An object associated with a user collects data that is stitched together and synchronized to provide post event analysis and training. Through an interface actions that occurred during an event can be recreated providing the viewer with information on what the user associated with the object was experiencing, where the user was looking, and how certain actions may have changed the outcome. Using the collected data, a virtual realty environment is created that can be manipulated to present alternative courses of action and outcomes.

BACKGROUND OF THE INVENTION Field of the Invention

Embodiments of the present invention relate, in general, tovirtual-reality media systems and more particularly to 360-degreevirtual reality media systems which capture and render for reviewreal-time dynamic events.

Relevant Background

Current virtual-reality (VR) recording (audio/video) systems (let alone360-degree-field-of-view systems) have been concerned neither with theheight nor weight of the VR recording device nor the physical impact ofeither factor (height/weight)—along with its related safety concern—whenthe VR device is associated with a user during a dynamic event. “Dynamicevent” is meant to mean any event in which an object, such as a helmet,is worn by a user, in which the object (and user) may be subject tophysical impacts incurred during the event, and in which the activity tobe debriefed takes place at such a pace that real-time debriefing,training or learning, generally, is impractical or impossible. Examplesof dynamic events include—but are by no means limited to—skiing, hockey,football, baseball, motocross, base-jumping, martial-arts training andthe like. This general lack of concern for height, weight and safetyaspects of VR devices in dynamic events has meant that state-of-the-art360-degree VR devices have not been widely used for dynamic-eventtraining or review.

A need thus exists for a data collection and VR system which is able tomeet these height/weight/safety constraints by maintaining a low profileor internal integration with respect to the helmet's or object'sexterior; capable of sustaining physical impacts to the object whichtypically occur during dynamic events; and operable to capture, andlater render, data from multiple vantage points and perspectives fordebriefing, training and learning of real-time-dynamic events. These andother deficiencies of the prior art are addressed by one or moreembodiments of the present invention.

Additional advantages and novel features of this invention shall be setforth in part in the description that follows, and in part will becomeapparent to those skilled in the art upon examination of the followingspecification or may be learned by the practice of the invention. Theadvantages of the invention may be realized and attained by means of theinstrumentalities, combinations, compositions, and methods particularlypointed out in the appended claims.

SUMMARY OF THE INVENTION

A 360-degree rendering of a user's visual environment during a dynamicfield of play can be captured, stored and rendered for laterexamination. Correlating the 360-degree rendering with a user's locationand motion along with the location and motion of other individuals andobjects in a field of play provides the user and others alike theability to review, critique, train and modify behavior to achieveimproved results.

In one embodiment of the present invention a dynamic event capture andrendering system includes two or more cameras affixed to an objectwearable by a user. Each camera is configured to capture video with thetwo or more cameras having overlapping fields of view so as to capture360-degrees of images surrounding the user. The system also includes aprocessor capable of executing instruction embodied as code and anon-transitory storage medium tangibly embodying such program ofinstructions.

These program of instructions include code for combining video from thetwo or more cameras so as to form a 360-degree rendering. That renderingincludes a central plane and a central vision point that can be alignedwith the visual field of the user wearing the object and the user'svisual fixation point. Instructions are also included to determine alocation of the object (and thereby the user associated with the object)as well as the motion of the object within the field of play.

The instructions, executed by the processor, thereafter synchronize the360-degree rendering with the location and the motion of the object andother forms of collected data and thereafter store the 360-degreerendering, the location, motion of the object, etc. for later analysis.

In other embodiments of the present invention, the system includeinstructions and related equipment to detect eye movement of the userrelative to the object (helmet in one example) so as to identify theuser's visual fixation point. Knowing the visual fixation point of theuser, additional code is included in another embodiment, to center thecentral vision point of the 360-degree rendering with the visualfixation point of the user.

In some embodiments the system of the present invention includes atransceiver that can be used to identify a presence of one or morenearby objects (other players), determine a relational location of eachnearby objects, and develop a spatial awareness of a local environmentsurrounding the object (player).

In another version of the present invention the transceivers can receivea signal from one or more transmitters affixed at a known locationrelative to the field of play (rink, field, court), and ascertain thelocation of the object (player) using trilateration or similartechniques. In yet another version an overhead video capture of theentire field of play can be correlated with data collected from eachsystem to determine the location of each object and nearby players atany point in time.

The data collected is combined, correlated and synchronized to form acontinuous 360-degree rendering from the point of view of a user(player) on the field of play. This rendering, or a select portion ofthat rendering, can thereafter be displayed on user interface such as amonitor, virtual reality goggles or the like. In another version of theinvention, the display of the 360-degree rendering of one user can alsoinclude select portions of the 360-degree rendering of one or moreadditional objects (user/player) along with the location of the one ormore additional objects on the field of play.

Each of these renderings and the data collected by the system of thepresent invention can, in another embodiment, form a virtual realty datastore from which program code can create a virtual reality environmentrelated to the field of play and the dynamic events captured. Thisinformation can include the location of the user and others on the fieldof play as well as events that warrant further analysis.

In some embodiments of the present invention patterns in the data areidentified such as key events, opportunities for improved performance orinconsistencies. Using artificial intelligence, the system can identifypoints in a field in play at which a different action may result in adifferent outcome. These insights can thereafter be presented to theuser, coach, trainer or the like for behavior modifications.

Another version of the present invention is a method for dynamic eventcapturing and rendering. Such a method includes the steps of capturingvideo, by each of two or more cameras that are affixed to an objectwearable by a user. The video from the two or more cameras haveoverlapping fields of view which capture the user's visual field theuser's visual fixation point.

The method continues by retrieving, from a non-transitory storagemedium, a program of instructions that includes a number of programcodes. A processer then continues the methodology by accessing capturedvideo from the two or more cameras and executing one or more of theprogram codes retrieved from the non-transitory storage medium.

The instructions found in the program codes cause the processor tocombine video from the two or more cameras to form a 360-degreerendering surrounding the user. This rendering includes a central planeand a central vision point. The instructions also determine a locationof the object and a motion of the object within a field of play andsynchronize the 360-degree rendering with the location and motion of theobject. Lastly the instructions direct the processor to store the360-degree rendering, the location and the motion of the object.

In some embodiments of the present invention the instructions of codemodify/align the central plane of the 360-degree rendering with a visualfield of the user. Other instructions detect eye movement of the userrelative to the object identifying a user visual fixation point andcenter and identify the central vision point of the 360-degree renderingabout the user visual fixation point. By doing so the exact point atwhich the user is looking in the 360-degree rendering at any point oftime can be determined and presented.

Another step in the process for capturing and rendering dynamic eventsdone by the processor is to identify a presence of one or more nearbyobjects, determine a relational location of each of the one or morenearby objects, and develop a spatial awareness of a local environmentsurrounding the object. This provides a picture or environment ofplayers surrounding the user.

The processors can also receive from a transceiver a signal from one ormore transmitters affixed at a known location relative to the field ofplay and ascertain the location of the object using trilateration basedon the signal received from the one or more transmitters. With thisinformation the processors can determine location and motion of theobject (user).

The transceivers found in the object in conjunction with the processorscan also transmit the captured data via a network to a remote locationfor storage and processing.

The information collected by the cameras and other sensors arecorrelated so that the 360-degree rendering, location, motion and audiofiles when combined present a combined and reconstructed field of playenvironment for analysis. Once combined the methodology can display on auser interface such as monitor or set of VR goggles a select portion ofthe 360-degree rendering along with the location of the first object onthe field of play. Moreover, any portion of the 360-degree renderingother objects (players) on the field of the play can be viewed at thesame point of time. These and other renderings, form a virtual realitythat can used to replay the field of play for analysis and training.

The features and advantages described in this disclosure and in thefollowing detailed description are not all-inclusive. Many additionalfeatures and advantages will be apparent to one of ordinary skill in therelevant art in view of the drawings, specification, and claims hereof.Moreover, it should be noted that the language used in the specificationhas been principally selected for readability and instructional purposesand may not have been selected to delineate or circumscribe theinventive subject matter; reference to the claims is necessary todetermine such inventive subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The aforementioned and other features and objects of the presentinvention and the manner of attaining them will become more apparent,and the invention itself will be best understood, by reference to thefollowing description of one or more embodiments taken in conjunctionwith the accompanying drawings, wherein:

FIG. 1 is a block diagram of an object for capturing and renderingdynamic events according to one embodiment of the present invention;

FIG. 2 is a logical block diagram of an application for capturing andrendering dynamic events according to one embodiment of the presentinvention;

FIG. 3A shows one version of a top, two-dimensional view of overlappingfields of view of a three-camera array associated with an object worn bya user in comparison to the visual field of view of the user, accordingto one embodiment of the present invention;

FIG. 3B shows a second version of a top, two-dimensional view ofoverlapping fields of view of a three-camera array associated with anobject worn by a user in comparison to the visual field of view of theuser, according to one embodiment of the present invention;

FIG. 3C is a rendition of one embodiment of an object of the presentinvention as a separate housing affixed to the top of a user's helmet;

FIG. 4 is a representative field of play having a plurality of playerseach associated with an object of the present invention and boundariesin which each user's location is determined using trilateration;

FIG. 5 is a high-level network diagram illustrating the distributenature of one embodiment of the present invention;

FIG. 6 is a flowchart of a process for capturing and storing a360-degree rendering of a dynamic event according to one embodiment ofthe present invention;

FIG. 7 is a flowchart of a process for retrieving, correlating andrendering select portions of 360-degree rendering, location and motionof a plurality of users in a field of play environment;

FIG. 8 presents a representative user interface providing multiple viewsas captured by one or more embodiments of the present invention;

FIG. 9 presents a visual depiction of a portion of a 360-degreerendering used for analysis and teaching according to one embodiment ofthe present invention.

The Figures depict embodiments of the present invention for purposes ofillustration only. Like numbers refer to like elements throughout. Inthe figures, the sizes of certain lines, layers, components, elements orfeatures may be exaggerated for clarity. One skilled in the art willreadily recognize from the following discussion that alternativeembodiments of the structures and methods illustrated herein may beemployed without departing from the principles of the inventiondescribed herein.

DESCRIPTION OF THE INVENTION

The capture and later rendering of a dynamic event in a field of playenvironment is hereafter described by way of example. A 360-degreerendering of a user's visual environment during a dynamic field of playcan be captured, stored and rendered for later examination. Correlatingthe 360-degree rendering with a user's location and motion, along withthe location and motion of other individuals and objects in a field ofplay, provides the user and others alike the ability to review,critique, train and modify behavior to achieve improved results.

Embodiments of the present invention are hereafter described in detailwith reference to the accompanying Figures. Although the invention hasbeen described and illustrated with a certain degree of particularity,it is understood that the present disclosure has been made only by wayof example and that numerous changes in the combination and arrangementof parts can be resorted to by those skilled in the art withoutdeparting from the spirit and scope of the invention.

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of exemplaryembodiments of the present invention as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the embodiments described hereincan be made without departing from the scope and spirit of theinvention. Also, descriptions of well-known functions and constructionsare omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings but are merely used by theinventor to enable a clear and consistent understanding of theinvention. Accordingly, it should be apparent to those skilled in theart that the following description of exemplary embodiments of thepresent invention are provided for illustration purpose only and not forthe purpose of limiting the invention as defined by the appended claimsand their equivalents.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. Thus, for example, reference to “a component surface”includes reference to one or more of such surfaces.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present), and B is false (or not present), A is false (or notpresent), and B is true (or present), and both A and B are true (orpresent).

As used herein, the term “substantially” it is meant that the recitedcharacteristic, parameter, or value need not be achieved exactly, butthat deviations or variations, including for example, tolerances,measurement error, measurement accuracy limitations and other factorsknown to those of skill in the art, may occur in amounts that do notpreclude the effect the characteristic was intended to provide.

It will be also understood that when an element is referred to as being“on,” “attached” to, “connected” to, “coupled” with, “contacting”,“mounted” etc., another element, it can be directly on, attached to,connected to, coupled with or contacting the other element orintervening elements may also be present. In contrast, when an elementis referred to as being, for example, “directly on,” “directly attached”to, “directly connected” to, “directly coupled” with or “directlycontacting” another element, there are no intervening elements present.It will also be appreciated by those of skill in the art that referencesto a structure or feature that is disposed “adjacent” another featuremay have portions that overlap or underlie the adjacent feature.

Spatially relative terms, such as “under,” “below,” “lower,” “over,”“upper” and the like, may be used herein for ease of description todescribe one element or feature's relationship to another element(s) orfeature(s) as illustrated in the figures. It will be understood that thespatially relative terms are intended to encompass differentorientations of a device in use or operation in addition to theorientation depicted in the figures. For example, if a device in thefigures is inverted, elements described as “under” or “beneath” otherelements or features would then be oriented “over” the other elements orfeatures. Thus, the exemplary term “under” can encompass both anorientation of “over” and “under”. The device may be otherwise oriented(rotated 90 degrees or at other orientations) and the spatially relativedescriptors used herein interpreted accordingly. Similarly, the terms“upwardly,” “downwardly,” “vertical,” “horizontal” and the like are usedherein for the purpose of explanation only unless specifically indicatedotherwise.

Unless otherwise defined below, all terms (including technical andscientific terms) used herein have the same meaning as commonlyunderstood by one of ordinary skill in the art to which this inventionbelongs. It will be further understood that terms, such as those definedin commonly used dictionaries, should be interpreted as having a meaningthat is consistent with their meaning in the context of thespecification and relevant art and should not be interpreted in anidealized or overly formal sense unless expressly so defined herein.Well-known functions or constructions may not be described in detail forbrevity and/or clarity.

As used herein, the term “fixation point” is meant to mean a central orfocused point within a user's field view. When a user focuses or looksat an object, their fixation point is that point on the object which isthe subject of interest.

As used herein, the term “field of play” is meant to mean a boundedregion in which one or more users engage in a dynamic actively. Insports a field of play can include a hockey rink, a football field, alacrosse field, a soccer field, or a basketball court. In otherenvironments a field of play may be tactical assault course orfirefighting training structure.

As used herein, the term “field of play environment” is meant to meancombined and correlated renderings of users, objects and occurrences ofa select field of play including a plurality of dynamic events.

Dynamic events are captured, stored and rendered for later analysis andtraining through overlapping field of view cameras worn by, andassociated with, a user. According to one embodiment of the presentinvention, two or more wide angle cameras are affixed to, or integratedwith, an object worn by a user with overlapping fields of view capturinga 360-degree rendering around the user. In one embodiment of the presentinvention, three cameras are assembled in an object which is mounted onthe top of helmet worn by the user during a particular activity. Eachcamera possesses a field of view so that when property positioned withinthe object a 360-degree continuous rendering surrounding the user iscaptured.

In other embodiments of the present invention, the cameras areintegrated into a helmet or head gear worn by the user. In the samemanner the fields of view of each camera are sufficient to overlap so asto continually capture the entire 360-degree image surrounding the user.

FIG. 1 is a high-level block diagram of a dynamic event capturing device100 according to one embodiment of the present invention. The deviceincludes, but is not limited to, a circuit board 110 or similarprocessing environment, two or more cameras 115 a, 115 b, 115 n, amicrophone array 120 and an eye movement and position detector 125.

The processing environment to the device includes a video interface 130,a clock 135, processor 140, battery 145 or similar power source, anaccelerometer and/or gyroscope 150, non-transitory memory 155, a networkinterface card 160 and a transceiver 170. The separation of the variouscomponents in the embodiments described herein should not be understoodas requiring such separation or each component in all embodiments. Itshould be understood that the described components may be integratedtogether in a single component or as separate components to achieve thedesired functionality. Additions, modifications, or omissions may bemade to the illustrated embodiment without departing from the scope ofthe disclosure.

While FIG. 1 illustrates an array of cameras 115, one video interface130, one processor 140, one network card 160, one transceiver 170, onemicrophone array 120, one eye detection device 125, and oneaccelerometer 150, the disclosure applies to a system architecturehaving one or more camera arrays, one or more video interfaces, one ormore possessors, one or more network cards, one or more microphonearrays, one or more eye detectors and one or more accelerometers. Inpractice more than one object may be networked together with variouscomponents of each object and their functionality distributed.

The camera array 115 may comprise a modular camera system configured tocapture raw video data that includes image frames. While in a preferredenvironment the camera array for each object includes two or morecameras, the number of cameras is not to be viewed as a limitingrequirement. A single camera capable of capturing a seamless 360-degreerendering would fully meet the objectives of the present invention andis indeed contemplated.

Each camera within the camera array 115 may capture video and/or audiodata and in the latter instance eliminate the need for a separatemicrophone array 120. Each camera may record in high definition (HD), 2Kvideo data, 4K+ video data or the like.

In other embodiments of the present invention, each camera may includeother sensors including, but not limited to a depth sensor, motionsensor, global positioning system sensor, Ultra-Wide Band receiver,accelerometers and the like. The camera may be fixedly mounted to theprocessing environment or fixed to an object worn by the user andcommunicatively coupled to the processing environment.

The cameras 115 may also be configured and positioned to capturedifferent fields of view and with differing degrees of fidelity. Forexample, a high-definition camera or cameras may be associated with aforward field of view while a lower definition or wide-angle camera maybe directed toward a rear facing field of view.

Each camera 115 a, 115 b . . . 115 n is communicatively coupled to avideo interface 130. In one embodiment each camera may be coupled to thevideo interface 130 via a wired line such as a HDMI cable, ethernetwire, universal serial bus cable, RCA cable, Firewire, or similar linkwhile in other embodiments one or more camera may be coupled to thevideo interface wirelessly via WIFI, Bluetooth or the like.

The video interface 130 may receive the raw video data and forward itfor processing or process the video internally. In other embodiments thevideo interface 130 may receive and aggregate the video streamsdescribing frames captured by each respective camera 115 a, 115 b . . .115 n. In other embodiments the video interface 130 working with theprocessor 140 may stitch overlapping video streams together using agraphic processor (not shown) or the like to form a cohesive singleimage stream encompassing a 360-degree view surrounding the user. In yetother embodiments the video interface 130, processor 140 and networkcard 160 may prepare video streams for aggregated or singulartransmission to a network server for additional processing includingstitching together overlapping fields of view.

In another embodiment the video interface 130 is a client-side devicethat is connected with one or more cameras 115 and/or a storage devicehaving one or more recordings. The client-side device may have one ormore interfaces within which a camera 115 or a storage device can beconnected and used to input recordings from the camera 115 or a storagedevice. The video interface 130 may also be connected to the network andupload the recordings from the one or more cameras 115 or the storagedevice to the virtual reality processing server and/or to the virtualreality data store.

The video interface 130, for example, may be a computer or laptop or maybe connected with a computer or a laptop. A camera or a storage devicemay be connected to the video interface via USB, FireWire, HDMI, etc.Alternatively, or additionally, the client device may include a storagedevice port such as, for example, an SD port, SD slot.

In another embodiment, the video interface may include a server-sideinterface.

The video interface 130 may receive recordings from one or more cameras115 or an aggregate recording regardless of camera type. The server-sidevideo interface may, for example, be hosted on a web server and/or aspart of the virtual reality processing server. The server-side videointerface, for example, may provide a number of data entry fields orwidgets to collect information such as, for example, the number ofcamera, the data compression used to save virtual reality content, thebrand of the camera, the serial number of the virtual reality, the modelof the camera, the type of audio being uploaded, the angular position ofthe camera modules, the files comprising the video and/or audio datafrom each individual camera, etc.

In other embodiments, the recordings received from the cameras 115 orthe video interface 130 may be uploaded to the virtual realityprocessing server and/or stored in the data store. The virtual realityprocessing server may receive raw video data and raw audio data from thevideo interface via the network. The virtual reality server may storethe data as virtual reality data. In other embodiments the virtualreality server may aggregate the video data captured by cameras to formvirtual reality streams and combine the virtual reality streams withaudio data to form virtual reality content that is stored in the datastore as virtual reality data.

The virtual reality processing server may include a system configured toaggregate raw video data and raw audio data to generate 3D video dataand 3D audio data, respectively. In some other embodiments, the virtualreality processing server may be implemented using a combination ofhardware and software. The virtual reality processing server, which mayreside in the cloud, may be configured for aggregating raw video datacaptured by two or more cameras including data from two or moredifferent objects. For example, the virtual reality processing servermay be configured to receive data from a first object and a one or moreadditional objects, determine parameters for each system of theplurality of camera systems, and aggregate data from two or more of theplurality of camera systems.

In other embodiments, the virtual reality processing server may create avirtual reality project from the virtual reality recordings uploaded tothe virtual reality data store. A virtual reality project may include,for example, virtual reality recordings, segments, and/or renderings.

Virtual reality renders may be created in any number of ways by thevirtual reality processing server. In some embodiments, the virtualreality processing server may identify a location and/or a timingassociated with each of the camera modules within a camera array. Thevirtual reality processing server may synchronize the various videoframes within a segment and/or recording based on locations and/ortimings associated with the camera modules.

The microphone array 120 may be associated with each camera or includeseparate components positioned on the object worn by the user. In oneembodiment of the present invention the microphones are directional andpositioned about the object to record and store audio file as well as adirectional source of the audio transmission. The microphone array 120may include different microphone systems and may include, like thecameras, separate processors and storage mechanisms.

The microphone array 120 can capture sound from different directions andstore raw audio data on a non-transitory storage medium. The data, likethe video files, can be processed internally on the object ortransmitted via the network card to a network server for laterprocessing. In some embodiments, the microphone array 120 may includesound field tetrahedral microphones following the principles ofambisonics, enabling reconstruction of sound from any arbitrarydirection. The position and configuration of the microphones can beindependent of cameras or in other embodiments, they may coexist.

In some embodiments, the cameras 115 may be mounted around or within acamera housing (e.g., a spherical housing or a housing with anothersuitable shape). The microphone array may include multiple microphonesmounted around the same camera housing, with each microphone located ina different position. The camera housing may act as a proxy for thehead-shadow sound-blocking properties of a human head.

Another component of the device for capturing and rendering dynamicevents is an eye movement detector 125 and tracker. Eye tracking is theprocess of measuring either the point of gaze (where one is looking) orthe motion of an eye relative to the head. In one embodiment videoimages of the eye are captured by a detection camera focused on theuser's eyes, from which the eye position is extracted. In otherembodiments light, typically infrared, is reflected from the eye andsensed by a video camera or some other specially designed opticalsensor. The information is then analyzed to extract eye rotation(movement) from changes in reflections. A still more sensitive method oftracking is to image features from inside the eye, such as the retinalblood vessels, and follow these features as the eye rotates. One or moreof these or similar techniques to capture eye movement and position isgathered and processed to identify the user's focus.

The device further includes an accelerometer and/or gyroscope to deducemovement (or non-movement) of the device. From a known starting orstatic position, data gained from a accelerometers in three dimensions,can determine a relative direction and velocity of the object. Usingdead reckoning a user's location within the field of play can be roughlydetermined. Moreover, the system can determine precise timing of eventswhich cause the user and object to abruptly change direction. A fall orimpact would register a significant acceleration. This data can becorrelated with the video and audio fields to recreate the field of playenvironment.

The accelerometer and/or gyroscope 150 associated with the presentinvention are further coupled to the video interface 130 and processor140 to facilitate video stabilization. As one of reasonable skill in therelevant art will appreciate, dynamic events such as hockey, footballand the like involve impacts and sudden changes of direction. Thiserratic, unpredictable movement can create a video which is unstable andunusable. The present invention stabilizes the 360-degree renderingusing data collected by the accelerometer and/or gyroscope to produce arendering which is stable despite erratic and even violent impacts.

A transceiver 170 is coupled to the system bus and can transmit datawirelessly to a receiver. In doing so real-time video and audio streamscan be captured apart from the object for later processing Thetransceiver can also be used to determine the location of theuser/object on the field of play using trilateration or time distance ofarrival techniques.

Trilateration is a technique that determines position based on distanceinformation from uniquely identifiable ranging radios. In trilateration,the position of a mobile object can be calculated using the knownpositions of multiple radio frequency reference beacons (anchors) andmeasurements of the distances between the mobile object and the anchors.The anchors can pinpoint the mobile object by geometrically forming fouror more spheres surrounding the anchors which intersect at a singlepoint, which is the location of the mobile object. Trilateration hasstrict infrastructure requirements, requiring at least three anchors fora 2D position and four anchors for a 3D position. In some embodimentsthe surface at the earth (field of play), can act as one sphere.

In another embodiment the location of the object on the field of viewcan be determined trilateration or TDOA. By having two or more fixedtransmitters each object can receive a predetermined signal from a knownlocation. Measuring the differences in time from each location providesa range or sphere on which the object must exist. Using the surface ofthe field of play as an additional sphere on which the object resides,the position of the object can be determined.

Time Distance of Arrival (“TDOA”) determines an object's location bymerely receiving broadcast signals rather than transmit and receive asin trilateration. In TDOA a plurality of nodes, such as in a Radiolocalization system, broadcast a signal at a precise time. The receivingnode receives two or more packets related to the same signal and noteseach time of arrival. Knowing the location of the transmitting nodes andthe different times that the same signal arrived at the receiving node,the receiving nodes location can be determined. When any two other nodesin the area perform a two-way ranging conversation, a node can overhearboth the request packet and the response packet and measures the timedifference of arrival of each. This time difference along with thelocations and location errors of these transmitters (which they includedin their signal) is used for updating current position of the eavesdropping node.

By using one or more these techniques or an overhead video captureanalysis, the location of the object can be determined in the field ofplay. As the user moves throughout the field of play the location of theobject, and thus the user, is determined by a location module. In thisembodiment of the present invention, an overhead video stream of thefield of play and the user's location on the field is recorded andsynchronized with the 360-degree rendering and visual fixation point. Insuch an embodiment each object can possess a unique RF beacon or visualmarker that can be observed/detected by an overhead sensor. The sensorthereafter communicates the data to the rendering system via thecommunication module.

FIG. 2 provides a logical depiction of processes by which a system 200for capturing and rendering dynamic events interact. As will beappreciated by one of reasonable skill in the relevant art, a portion ofthe present invention can be embodied in firmware or software.Instructions embodied as program code or software can be stored on anon-transitory storage medium and executed by a processor and recognizeda separate functional modules. The storage medium can further storevideo data from the one or more cameras, audio files, positionalinformation and eye tracking data accessed by the functional moduleswhich thereafter manipulates, modifies and renders the data.

While FIG. 2 illustrates, in one embodiment, a video combination module205, an eye detection module 200, a motion detection module 220, anaudio module 225, a location module 230, a communication module 240, auser interface module 250 and a tagging module. Other modules, datacollected by the cameras, microphones and other sensors, can befashioned and implemented to collect, capture, store and render dynamicevents. One of reasonable skill in the relevant art will recognize thatthe names, separation or configuration of the modules shown in FIG. 2are illustrative only and not intended to limit the scope of theinvention.

One objective of the present invention is the capture and rendering of a360-degree recreation of a dynamic event. In doing so, data gathered bysensors such as a camera array, microphone array and transceivers arestored in data store 260 or comparable non-transitory storage medium.Instructions, stored in memory 270 and executed by the processor 280,receive and act on the data to produce a useful rendering of a pastevent or series of events.

A video combination module 205, in one embodiment, joins otherwiseseparate data streams gathered by cameras affixed to the object into acohesive 360-degree rendering. The rendering is centered upon the objectworn by the user and modified to identity a central plan and a centralvision point. With additional reference to FIGS. 3A and 3B, each camera115 a, 115 b, 115 n includes a defined field of view 310. The objectpresented in FIG. 3A includes three cameras 115, each with a field ofview of 150 degrees. In this illustration a user is facing toward thetop of the figure and the object 305 is a helmet worn by the user with afirst camera 115 a and a second camera 115 b positioned near the forwardleft and right temples of the user. A third camera 115 c is positionednear the rear of the helmet 305. The fields of view 310 of each camera115 are adjusted to equally overlap 330 by approximately 15 degrees.

Data collected from each camera is provided to the video combinationmodule to craft a seamless 360-degree rendering surrounding the user.Data (images) found in each overlap zone 350 is common, in theillustration of FIG. 3, to two cameras. These common objects, horizons,disturbances, and the like can be used as data points by which tomodify, align and combine the adjoining video streams. While each camerais similar and may be of the same type, data collected by each will notbe identical. Minor modifications in image processing must be corrected(modified) to stitch the images together. Using graphic and centralprocessing techniques the video data stream is configured to provide a360-degree rendering surrounding the object.

Apart from the stitching of the video images captured by each camera,the 360-degree rendering itself is, in one embodiment, adjusted toconform with the visual field 366 of a user. Recall, each of the camerasin the embodiment shown in FIG. 3 are positioned on/in the object tocreate overlapping fields of view. While two of the cameras 115 a, 115 bare somewhat proximate to the human eyes of the user, they are notlocated in the same location as the user's eyes. Indeed, the cameras andthe eyes of the user may be in a different plane or the plane may be notaligned with visual field of the user. The 360-degree rendering isadjusted to provide a central plane aligned with the field of view ofthe user as well as a central vison point 370 substantially in front ofthe user. In one embodiment of the present invention, the central visionpoint 370 of the 360-degree rendering of a user wearing the object ashelmet will fall on a substantially perpendicular line extending outwardfrom the midpoint between the two forward cameras.

The object presented in FIG. 3B also includes three cameras 115, eachwith a field of view of 150 degrees but arranged to orient a singlecamera forward and two to the rear. Again, the user is facing toward thetop of the figure and the object 305 is a helmet or the like worn by theuser. The first camera 115 a is positioned centrally atop the user'seyes. The second and third cameras 115 b, 115 c are positioned near therear quadrants of the helmet 305. The fields of view 310 of each camera115 are adjusted to equally overlap 330 by approximately 15 degrees.

In one embodiment, as shown in FIG. 3C, the object 305 of the presentinvention affixed to the top a player's helmet 380. In this embodimentthe housing is triangular in shape with a wide-angle camera 115positioned at each vertex. Audio sensors are similarly distributed inthe housing with the remaining components housed within. The objectweighs approximately 250 grams with a profile height of 22 mm andmaximum breadth of 79 mm. The device can operate independently oninternal power for up to 3 hours with a storage capacity of 64 GB.Cameras are synchronized with a common time stamp as is gyroscopic andaccelerometer data. This data can further be used to recognize a restingstate as would occur when a player as on the sideline and not activelyengaged in the field of play. In such instances the device can powerdown to a standby mode to conserve power and reactive upon therecognition of certain motion data. Control interfaces are included onthe surface of the housing for ease of control.

In one embodiment the size of the object is minimized by having a singleforward-facing camera. In some instances, the complexity of the devicecan be lessened by using a single wide-angle camera to capture theforward aspect of a user's field of view. Certain activities areunidirectional while others are multidirectional. By modifying the scopeof coverage weight, cost and complexity can be managed to best fit thedemand to capture associated dynamic events.

In an embodiment in which the object worn by the user is a helmet and towhich the cameras are affixed or integrated within, the field of view ofeach camera rotates as the head of the user moves. The visual fixationpoint of the user however is not always consistent with the position ofa user's head.

Humans have a visual fixation point of view in binocular vision ofapproximately 7 degrees. By rotating one's head and/or moving one's eyesa user can quickly move their visual fixation point to gather data onwhich to react. The movement of, and accuracy of, a person's head ascompared to movement of eyes is dramatic. Eyes are capable of quick andaccurate targeting on which to focus. As the head rotates, the eyescapture an image of interest first in peripheral vison and then withinthe visual fixation point for interpretation.

Eye tracking refers to the process of measuring where a person looks,also known as a point of gaze. These measurements are carried out, inone embodiment, by eye detection and tracker module 210. This modulerecords the position of the eyes and the movements they make inconjunction with the movement of the head as determined by the motiondetection module 220. In one embodiment of the present invention anoptical method for measuring eye motion is used. In such a method light,typically infrared, is reflected from the eye and sensed by a videocamera or some other specially designed optical sensor.

The present invention captures both the movement of the object (helmet)as well as movement of eye with the motion detection 220 and eyedetection module 210, respectively. Using accelerometers within thedevice and a static point of reference, motion of the object iscollected and combined with movement of eyes to identify what the useris looking at for any point in time.

An audio module 225 captures and refines audio files detected by themicrophone array. As with a user's ears, the microphone array 120 inconcert with the audio module 225 of the present invention can determinethe relative direction of a particular sound. In the same manner as thehuman brain, microphones positioned a finite distance apart register hesame sound at slightly different times. These differences can be used todetermine relative bearing.

Results from each module are synchronized and presented in a userinterface as a viewing system enabling a third party or the userthemselves to view the dynamic events as if they were in the field ofplay. In another embodiment of the present invention, a tagging module225 marks significant events in the data stream. Events worth taggingmay be predetermined, manually entered, or determined by the moduleitself.

The viewing system may include or use a computing device to decode andrender a stream of 3D video data on a virtual reality display device.The viewing system may also decode and render a stream of 3D audio dataon an audio reproduction device (e.g., a headphone or other suitablespeaker devices). The viewing system may include a virtual realitydisplay configured to render the 3D video data and the audioreproduction device configured to render the 3D audio data. The systemmay be coupled to the network and a user may interact with the viewingsystem to customize his/her experience.

The viewing system may also include a web viewing device, a user device,a virtual reality device, and/or a console. Various other virtualreality devices may request and/or receive virtual reality content fromthe virtual reality content delivery network. The web viewing device mayinclude a computer, a laptop, or a tablet. The user device may include asmartphone or a tablet that may include a virtual reality application toplay back virtual reality content and/or may require the use of avirtual reality device in conjunction with the smartphone such as, forexample.

The viewing system may also track a head orientation of a user. Forexample, the viewing system may include one or more accelerometers orgyroscopes used to detect a change in the orientation of the user'shead. The viewing system may decode and render the stream of 3D videodata on a virtual reality display device and the stream of 3D audio dataon a speaker system based on the head orientation of the user. As theuser changes his or her head orientation, the viewing system may adjustthe rendering of the 3D video data and 3D audio data based on changes ofthe user's head orientation.

The viewing system may provide an immersive viewing experience to theuser. For example, the viewing system may include a virtual realitydisplay device that has a wide field of view so that the user viewingthe virtual reality content feels like he or she is surrounded by thevirtual reality content in a manner similar to in a real-lifeenvironment. A complete 360-degree view of the scene is provided to theuser, and the user may view the field of play in any direction. As theuser moves his or her head, the view is modified to match what the userwould see as if he or she were moving his or her head in the real world.Additionally, 3D surrounding sound may be provided to the user based onthe user's head orientation to augment the immersive 3D viewingexperience. For example, if a player in an immersive movie is currentlybehind the user, the player's voice may appear to be emanating frombehind the user.

The communication module 240 not only receives information from varioussensors but also enables the interface module to present data from morethan one object at a time. In such a manner, data from multiple objects(users) can be displayed side-by-side to ascertain the position, actionsand view of teammates.

The communication module 240 may further transmit data to any of theentities of the system. Similarly, the communication module 240 mayreceive data from any of the components. The communication module 240may include one or more Ethernet switches for receiving the processedvirtual reality video data from the network and the raw audio data fromone or more objects. The communication module 240 may be softwareincluding routines for handling communications between the virtualreality processing server and other components of the virtual realityprocessing server.

In some embodiments, the communication module 240 receives data fromcomponents of the virtual reality processing server and stores the datain memory 270 or the data store 260. For example, the communicationmodule 240 may receive virtual reality content from the videocombination module 205 and store the virtual reality content in the datastore 260 as virtual reality data. In some embodiments, thecommunication module 260 retrieves data from the memory 270 or datastore 260 and sends the data to one or more appropriate components ofthe virtual reality processing server.

To better appreciate the features of the present invention, consider thefollowing example. Consider a hockey team in which the players of oneteam each wear a helmets fitted with the dynamic event capturing systemof the present invention. As shown in FIG. 4, at each corner of the rink410 is an RF transmitter 420 transmitting a predetermine signalpossessing a common time reference.

As the match begins each player fitted with the object 430 of thepresent invention begins to collect data (video, audio, location,motion, etc.). In one embodiment the video, location, auditory, andmotion data is retained internally and downloaded at the conclusion ofthe match. In another embodiment the data is streamed continuously fromeach player to a common network server at which the data is processed.

Each helmet fitted with the object 430 of the present invention includesan optical eye tracking system to ascertain where the user (player) waslooking during the match relative to the helmet. Events such asfaceoffs, penalties, goals and the like are identified (tagged) as keyevents for later analysis.

Upon completion of the match the data from each camera is stitchedtogether and aligned with the field of view of the player. The centralvision point of the 360-degree rendering is further aligned with eachuser's visual fixation point so that not only can the rendering show avideo of everything around the player at any point in time but where, atthat point of time, the player was looking. This information issynchronized with audio and motion data to determine if any calls,orders, distractions or impacts may have been experienced.

As the playback progresses it can be stopped at any point in time or ata tagged event to review where the player was looking and review whataction(s) the player initiated. Not only can the coach or trainer seewhat a particular player was seeing but they can also alter the point ofview to look where the player was not looking.

For example, a player may have heard a call or distraction and lookedtoward the baseline or bench at a critical time in a play. The coach canreview the player's action and point out that the player should havebeen looking cross court for a pending pass or similar action from ateammate.

The invention also enables different points of view to be consideredconcurrently. The video streams from two players can be viewedside-by-side to determine where each player was looking and what clueswere presented to each other. Excessive attention to a defensive playeror a lack of attention to the location of the puck can be observed andcorrected.

The location of each player on the field of play as well as theenvironment in which the match is played can also be provided. As theviewpoint of each player can be observed and modified, the entiredynamic field of play can be reconstructed offering players and coachesalike an opportunity to analyze tactics, strategies, player positioningand actions.

As described above, some embodiments of the present inventionincorporate a virtual reality processor server and interface.Collectively a virtual realty system, one embodiment of the presentinvention captures real world data in the form of video, audio andpositional information to craft a virtual environment that provides arealistic field of play in which players and coaches can manipulatedetails of the environment and provide opportunities to modify or refinea skill set.

FIG. 5 is a high-level network diagram providing, according to oneembodiment of the present invention, a network topology. Objects 510providing a platform on which to gather video, audio and positional dataare communicatively coupled to a network 520. The network may be thepublic Internet or be a private intranet environment. In one version ofthe present invention raw data is conveyed through the network 520 to aremote video (data) processing system 530 or server. Inputs from variouscameras, audio streams and the like can be synchronized, aligned andstitched together to provide a compressive rendering of the field ofplay.

In some embodiments, information from the video processing is gatheredby a virtual reality system 540. The virtual reality system creates athree-dimensional rendering of the field of play based on collecteddata. The virtual field of play can be manipulated and presented tousers through a user interface, virtual reality googles or the like.

The processing of raw data by the video processing system 530 and thevirtual reality system 540 can be cloud based using off-site servers.Using cloud processing capability, the client-based systems forinterfacing with the data, real or virtual, can be minimized. In otherembodiments video processing can be remote yet located at a secure(private) location. Using the network 520 as a means for transport,data, raw and virtual, can be conveyed to a remote location at whichdata is processed to its final form and presented through an appropriateuser interface 550.

The methodology associated with presenting the dynamic renderings of thepresent invention in its final form begins with capturing streamingvideo, as illustrated in the flowchart presented in FIG. 6. In thedescription that follows it will be understood that each block of theflowchart illustrations, and combinations of blocks in the flowchartillustrations, can be implemented by computer program instructions.These computer program instructions may be loaded onto a computer orother programmable apparatus to produce a machine such that theinstructions that execute on the computer or other programmableapparatus create means for implementing the functions specified in theflowchart block or blocks. These computer program instructions may alsobe stored in a computer-readable memory that can direct a computer orother programmable apparatus to function in a particular manner suchthat the instructions stored in the computer-readable memory produce anarticle of manufacture including instruction means that implement thefunction specified in the flowchart block or blocks. The computerprogram instructions may also be loaded onto a computer or otherprogrammable apparatus to cause a series of operational steps to beperformed in the computer or on the other programmable apparatus toproduce a computer implemented process such that the instructions thatexecute on the computer or other programmable apparatus provide stepsfor implementing the functions specified in the flowchart block orblocks.

Accordingly, blocks of the flowchart illustrations support combinationsof means for performing the specified functions and combinations ofsteps for performing the specified functions. It will also be understoodthat each block of the flowchart illustrations, and combinations ofblocks in the flowchart illustrations, can be implemented by specialpurpose hardware-based computer systems that perform the specifiedfunctions or steps, or combinations of special purpose hardware andcomputer instructions.

Some portions of this specification are presented in terms of algorithmsor symbolic representations of operations on data stored as bits orbinary digital signals within a machine memory (e.g., a computermemory). These algorithms or symbolic representations are examples oftechniques used by those of ordinary skill in the data processing artsto convey the substance of their work to others skilled in the art. Asused herein, an “algorithm” is a self-consistent sequence of operationsor similar processing leading to a desired result. In this context,algorithms and operations involve the manipulation of informationelements. Typically, but not necessarily, such elements may take theform of electrical, magnetic, or optical signals capable of beingstored, accessed, transferred, combined, compared, or otherwisemanipulated by a machine. It is convenient at times, principally forreasons of common usage, to refer to such signals using words such as“data,” “content,” “bits,” “values,” “elements,” “symbols,”“characters,” “terms,” “numbers,” “numerals,” “words”, or the like.These specific words, however, are merely convenient labels and are tobe associated with appropriate information elements.

Capturing, processing and rendering dynamic events with a 360-degreefield of play view starts 605 with associating 610 two or more cameraswith an object worn by a user. Each camera includes a field of viewpositioned 615 to create an overlapping 360-panarama surrounding theuser. Video from each camera is combined 620 forming a 360-degreerendering in which a central vision point is designated 625. The centralvision point is a point identified as being directly in front of theuser. In a version of the present invention in which the object ismounted on or incorporated into a helmet, the central vision point ofthe 360-degree rendering would be determined by which way a user wouldbe facing when wearing the helmet. Assuming the helmet is well fit to auser, the central vision point of the 360-degree rendering willsubstantially coexist with a point directly in front of a user's face.

The process continues by identifying 630 a visual fixation point of theuser. From a static position a human possess approximately a 210-degreeforward-facing visual field. That field includes a vertical range ofapproximately 150 degrees. Within that vast range the fixation point, orthe point at which humans possess fine resolution, is approximately 6-7degrees. Eyes can move ±45 degrees in azimuth and ±40 degrees elevation(35 degrees in elevation, 45 degrees in depression) at a rate of morethan 100 degrees/second. Accordingly, knowing the direction a player wasfacing provides limited information as to where the player is looking.The present invention, in one embodiment, identifies the visual fixationpoint of the user and aligns 635 the central vision point of the360-degree rendering with the visual fixation point.

As motion, impacts, and adjustments in the object change with respect tothe user, the system updates 640 and confirms whether the visualfixation point, and the central vision point are aligned. When they arenot the process for aligned is reinitiated.

Location and motion of the object on the field of play is alsodetermined 645. Using techniques described herein and known one ofreasonable skill in the relevant art, the location of the object (andthus user) and the objects motion, velocity, movement, is gathered andsynchronized 650 with the 360-degree rendering. This visual rendering,along with the objects location motion and related data is stored 660 ina non-transitory storage medium completing 695 the process.

Turning with additional reference to FIG. 7, data stored on anon-transitory storage medium is retrieved 710 and correlated 720. Thecorrelated rendering data, motion, location and audio information formsa field of play environment in which the object exists. Through a userinterface such as a monitor or a set of virtual reality goggles a usercan view the field of play as experienced by a player can manipulate 730the user's fixation point to observer any point in the field of playenvironment at any time during the match.

FIG. 8 provides a representative a user interface rendering of datacollected by two or more objects of the present invention. In theexample shown in FIG. 8, three players in a hockey game equipped with anobject of the present invention capture data during a match. Uponcompletion of the match data from each was collected, processed andsynchronized along with a general video of the entire field of play. Theuser interface shown presents, on a laptop computer 810, the field ofplay 820, showing the location and movement of each player on the right,and the first person view 830 of each player on the left. The view ofeach player is combined with data identifying the other players as wellother information pertinent to the match. In this example, goalie is avital player in the action depicted yet the goalie's point of view isnot presented. As the video of played and the match reviewed, eachplayer's perspective can be altered. For example, from player 1's pointof view 833, player 1 840, who is looking at the goalie 845, can alsosee players 2 860 who may have open shot to the goal. Player 3 850 isfocused 832 on the goal and not cognizant of player 2 860. Player 2 860sees 831 sees player 3 850 and is not aware of player 1 840. The presentinvention provides the ability to stop an event and view that event froma different perspective to improve performance and decision making.

FIG. 9 provides another example of the invention's ability to providefeedback and analysis of a dynamic event. FIG. 9 shows two portions of aplayer's 360-degree rendering at the same instant in time. The rightmost rendering 910 is a forward-looking perspective with the left mostrendering 950 being a view over the player's left shoulder. In thisinstance the player has the puck and is looking to pass to the player915 in front of the goal 920. The player is looking at the end of thehockey stick as illustrated by the players visual fixation point 940.The invention, through data collected, identifies that player as thecenter 925. As the event unfolds the time it takes for the player toreact 930 is shown in the upper left corner. Had the player looked tothe left he/she would see a defensive player 955 charging to his/herposition emphasizing the need to react quickly. These types of scenarioscan be captured and replayed to provide a player with insight andexperience in real world environments thereby improving performance.

Another aspect of the present invention is to identify key events in arendering for later analysis and review. According to one embodimentpredetermined events can be marked or tagged in real time in therecorded data. Events such as the beginning of a match, penalties,goals, power plays, time outs, etc. can result in a marking of therecorded data to enable for quick access to important aspects of amatch. In another embodiment audio analysis can register a whistle froman official as a tagged event or the accelerometers can register animpact or fall.

In yet another embodiment the tagging module can analyze the 360-degreerendering to identify or tag events that would otherwise go unnoticed.Using historical data, the tagging module can learn formations ormovements that present learning opportunities. For example, a certainconfiguration and placement of offensive and defensive players maywarrant a particular approach. That configuration may warrant a taggingin the data. Moreover, a missed opportunity to execute a play orstrategy may also warrant a tagging event. As more data is collected thesystem learns to tag similar event through artificial intelligence. Thesystem possesses the ability to rationalize and take actions that havethe best chance of achieving a specific goal of identifying key learningmoments. Upon determining that the current instructions are inadequateto meet that goal, the module can automatically learn from and adapt tonew data without being assisted by humans.

The dynamic event capturing and rendering system of the presentinvention can enable players, coaches and officials alike relive amatch, game or any dynamic event. By associating an object of thepresent invention capable of capturing video, audio, and positional datawith a user on a field of play, a 360-degree rendering of that field ofplay and match environment from a user's (player's) perspective can begained. Not only can a player, coach or official look a back at what hashappened, but they can see the match from the user's perspective as wellas be provided a view from where the user was not looking. Key aspectsof a player's or official's actions can be identified and improved.

For example, a coach may playback a missed opportunity and show a playerthat instead of looking at this player, he/she should have been lookingat a different area of the rink, earth or field. Or an official mightsee that based on the action, he/she should be positioned in thisportion of the rink, earth or field and looking in this area to identifyinfractions.

Moreover, the presentation invention can be used as a virtualenvironment to modify the field of play, yet provide the user with arealistic environment. The system can learn from itself to identify keyevents and missed opportunities to be later reviewed.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs for asystem and a process for dynamic event capturing and rendering throughthe disclosed principles herein. Thus, while particular embodiments andapplications have been illustrated and described, it is to be understoodthat the disclosed embodiments are not limited to the preciseconstruction and components disclosed herein. Various modifications,changes and variations, which will be apparent to those skilled in theart, may be made in the arrangement, operation and details of the methodand apparatus disclosed herein without departing from the spirit andscope defined in the appended claims. Indeed, while the examples hereinhave been provided in the context of a hockey match, the presentinvention is equally applicable to other environments such as football,lacrosse, basketball, tactical training environments, and the like.

Likewise, the particular naming and division of the modules, managers,functions, systems, engines, layers, features, attributes,methodologies, and other aspects are not mandatory or significant, andthe mechanisms that implement the invention or its features may havedifferent names, divisions, and/or formats. Furthermore, as will beapparent to one of ordinary skill in the relevant art, the modules,managers, functions, systems, engines, layers, features, attributes,methodologies, and other aspects of the invention can be implemented assoftware, hardware, firmware, or any combination of the three. Ofcourse, wherever a component of the present invention is implemented assoftware, the component can be implemented as a script, as a standaloneprogram, as part of a larger program, as a plurality of separate scriptsand/or programs, as a statically or dynamically linked library, as akernel loadable module, as a device driver, and/or in every and anyother way known now or in the future to those of skill in the art ofcomputer programming. Additionally, the present invention is in no waylimited to implementation in any specific programming language, or forany specific operating system or environment.

Generally, program modules include routines, programs, objects,components, data structures and the like that perform particular tasksor implement particular abstract data types. Moreover, those skilled inthe art will appreciate that the invention can be practiced with othercomputer system configurations, including hand-held devices,multi-processor systems, microprocessor-based or programmable consumerelectronics, network PCs, minicomputers, mainframe computers, and thelike. As discussed herein, the invention may also be practiced indistributed computing environments where tasks are performed by remoteprocessing devices that are linked through a communications network. Ina distributed computing environment, program modules may be located inboth local and remote memory storage devices.

An exemplary system for implementing the invention includes ageneral-purpose computing device such as the form of a conventionalpersonal computer, server, or the like, including a processing unit, asystem memory, and a system bus that couples various system components,including the system memory to the processing unit. The system bus maybe any of several types of bus structures including a memory bus ormemory controller, a peripheral bus, and a local bus using any of avariety of bus architectures. The system memory generally includesread-only memory (ROM) and random access memory (RAM). A basicinput/output system (BIOS), containing the basic routines that help totransfer information between elements within the personal computer, suchas during start-up, is stored in ROM. The personal computer may furtherinclude a hard disk drive for reading from and writing to a hard disk, amagnetic disk drive for reading from or writing to a removable magneticdisk. The hard disk drive and magnetic disk drive are connected to thesystem bus by a hard disk drive interface and a magnetic disk driveinterface, respectively. The drives and their associatedcomputer-readable media provide non-volatile storage of computerreadable instructions, data structures, program modules and other datafor the personal computer. Although the exemplary environment describedherein employs a hard disk and a removable magnetic disk, it should beappreciated by those skilled in the art that other types of computerreadable media which can store data that is accessible by a computer mayalso be used in the exemplary operating environment.

Embodiments of the present invention as have been herein described maybe implemented with reference to various wireless networks and theirassociated communication devices. Networks can also include mainframecomputers or servers, such as a gateway computer or application server(which may access a data repository). A gateway computer serves as apoint of entry into each network. The gateway may be coupled to anothernetwork by means of a communications link. The gateway may also bedirectly coupled to one or more devices using a communications link.Further, the gateway may be indirectly coupled to one or more devices.The gateway computer may also be coupled to a storage device such asdata repository.

An implementation of the present invention may also be executed in a Webenvironment, where software installation packages are downloaded using aprotocol such as the HyperText Transfer Protocol (HTTP) from a Webserver to one or more target computers (devices, objects) that areconnected through the Internet. Alternatively, an implementation of thepresent invention may be executing in other non-Web networkingenvironments (using the Internet, a corporate intranet or extranet, orany other network) where software packages are distributed forinstallation using techniques such as Remote Method Invocation (“RMI”)or Common Object Request Broker Architecture (“CORBA”). Configurationsfor the environment include a client/server network, as well as amulti-tier environment. Furthermore, it may happen that the client andserver of a particular installation both reside in the same physicaldevice, in which case a network connection is not required.

While there have been described above the principles of the presentinvention in conjunction with a device and associated methodology forcapture and rendering of a dynamic event, it is to be clearly understoodthat the foregoing description is made only by way of example and not asa limitation to the scope of the invention. Particularly, it isrecognized that the teachings of the foregoing disclosure will suggestother modifications to those persons skilled in the relevant art. Suchmodifications may involve other features that are already known per seand which may be used instead of or in addition to features alreadydescribed herein. Although claims have been formulated in thisapplication to particular combinations of features, it should beunderstood that the scope of the disclosure herein also includes anynovel feature or any novel combination of features disclosed eitherexplicitly or implicitly or any generalization or modification thereofwhich would be apparent to persons skilled in the relevant art, whetheror not such relates to the same invention as presently claimed in anyclaim and whether or not it mitigates any or all of the same technicalproblems as confronted by the present invention. The Applicant herebyreserves the right to formulate new claims to such features and/orcombinations of such features during the prosecution of the presentapplication or of any further application derived therefrom.

1. A dynamic event capture and rendering system, comprising anon-transitory storage medium tangibly embodying a program ofinstructions wherein said program of instruction comprises a pluralityof program codes; an object wearable by a user; one or more sensorsconfigured to collect location and motion data of the object within afield of play, location and motion data of one or more other objectswithin the field of play, and a fixation point of the user; two or morecameras affixed to the object, the two or more cameras havingoverlapping fields of view, wherein the two or more cameras areconfigured to capture a panoramic 360-degree rendering around the user,wherein the panoramic 360-degree rendering includes the field of playand the one or more other objects in the field of play; and at least oneprocessor communicatively coupled to the non-transitory storage mediumand the two or more cameras and configured to execute one or more of theprogram codes, said program codes including program code for combiningvideo from the two or more cameras forming the 360-degree renderingwherein the rendering includes a central plane of the field of play, acentral vision point of the user, the fixation point of the user, andwherein the 360-degree rendering is independent of the central visionpoint and the fixation point, program code for determining a location ofthe object and a motion of the object within the field of play, programcode for determining and correlating a location of the other objectsrelative to the object and a motion of the other objects relative to theobject, both within the field of play, program code for synchronizingthe 360-degree rendering with the location and the motion of the objectand the location and motion of the other objects, program code forselectively displaying the location and the motion of the object, thecentral vision point of the user, the fixation point of the user, andthe location and motion of the other objects, in the field of play, andprogram code for storing the 360-degree rendering and data with respectto the location and the motion of the object and the location and motionof the other objects.
 2. The dynamic event capture and rendering systemof claim 1, wherein the object is a helmet.
 3. The dynamic event captureand rendering system of claim 1, wherein the object is a low-profilehousing affixed to a helmet.
 4. The dynamic event capture and renderingsystem of claim 1, further comprising program code for modifying thecentral plane of the 360-degree rendering to align with a visual fieldof the user.
 5. The dynamic event capture and rendering system of claim1, further comprising program code for detecting eye movement of theuser relative to the object identifying the fixation point of the user.6. (canceled)
 7. The dynamic event capture and rendering system of claim5, further comprising program code for identifying the fixation point inthe 360-degree rendering.
 8. The dynamic event capture and renderingsystem of claim 1, further comprising a transceiver.
 9. The dynamicevent capture and rendering system of claim 8, wherein determiningincludes identifying a presence of one or more nearby objects,determining a relational location of each of the one or more nearbyobjects, and developing a spatial awareness of a local environmentsurrounding the object including the one or more nearby objects.
 10. Thedynamic event capture and rendering system of claim 8, whereindetermining includes receiving a signal from one or more transmittersaffixed at a known location relative to the field of play, andascertaining the location of the object using trilateration based on thesignal received from the one or more transmitters.
 11. The dynamic eventcapture and rendering system of claim 8, further comprising program codefor transmitting the 360-degree rendering, the location, and the motionof the object.
 12. The dynamic event capture and rendering system ofclaim 1, wherein determining includes correlating the location of theobject with an overhead video rendering of the field of play.
 13. Thedynamic event capture and rendering system of claim 1, furthercomprising program code for displaying on a user interface a selectportion of the 360-degree rendering of the object as a first objectalong with the location of the first object on the field of play. 14.The dynamic event capture and rendering system of claim 13, furthercomprising program code for displaying on a user interface a selectportion of the 360-degree rendering of the one or more other objectsalong with the location of the one or more other objects on the field ofplay, the location of the first object on the field of play and theselect portion of the 360-degree rendering of the first object.
 15. Thedynamic event capture and rendering system of claim 1, furthercomprising a virtual realty data store and program code for a virtualreality environment related to the field of play.
 16. The dynamic eventcapture and rendering system of claim 1, further comprising program codeof integrating the 360-degree rendering of the object along with thelocation of the object on the field of play with a virtual realityenvironment related to the field or play.
 17. A method for dynamic eventcapturing and rendering, the method comprising: collecting, by one ormore sensors, location and motion data of an object wearable by a userwithin a field of play, location and motion data of one or more otherobjects within the field of play, and a fixation point of the user;capturing, by each of two or more cameras having overlapping fields ofview and affixed to the object, a panoramic 360-degree rendering aroundthe user, wherein the panoramic 360-degree rendering includes the fieldof play and the one or more other objects in the field of play;retrieving, from a non-transitory storage medium, a program ofinstructions wherein said program of instruction comprises a pluralityof program codes; and by at least one processor, accessing capturedvideo from the two or more cameras and executing one or more of theprogram codes retrieved from the non-transitory storage medium, saidprogram codes directing the one or more processors to combine video fromthe two or more cameras to form the 360-degree rendering wherein therendering includes a central plane of the field of play, a centralvision point of the user, and the fixation point of the user and whereinthe 360-degree rendering is independent of the central vision point andthe fixation point, determine a location of the object and a motion ofthe object within the field of play, determine and correlate a locationof the other objects relative to the object and a motion of the otherobjects relative to the object, both within the field of play,synchronize the 360-degree rendering with the location and the motion ofthe object and the location and motion of the other objects, display,selectively, the location and the motion of the object, the centralvision point of the user, the fixation point of the user, and thelocation and motion of the other objects, in the field of play, andstore the 360-degree rendering and data with respect to, the locationand the motion of the object and the location and motion of the otherobjects.
 18. The method for dynamic event capturing and rendering ofclaim 17, wherein the one or more processors modify the central plane ofthe 360-degree rendering to align with a visual field of the user. 19.The method for dynamic event capturing and rendering of claim 17,wherein the one or more processors detect eye movement of the userrelative to the object identifying the user visual fixation point. 20.(canceled)
 21. The method for dynamic event capturing and rendering ofclaim 19, wherein the one or more processors identify the user visualfixation point in the 360-degree rendering.
 22. The method for dynamicevent capturing and rendering of claim 17, wherein the one or moreprocessors identify a presence of one or more nearby objects, determinea relational location of each of the one or more nearby objects with theobject, and develop a spatial awareness of a local environmentsurrounding the object including the one or more nearby objects.
 23. Themethod for dynamic event capturing and rendering of claim 17, whereinthe one or more processors receive from a transceiver a signal from oneor more transmitters affixed at a known location relative to the fieldof play and ascertain the location of the object using trilaterationbased on the signal received from the one or more transmitters.
 24. Themethod for dynamic event capturing and rendering of claim 17, furthercomprising transmitting, by a transceiver, the 360-degree rendering, thelocation, and the motion of the object.
 25. The method for dynamic eventcapturing and rendering of claim 17, further comprising transmitting, bya transceiver, the captured video.
 26. The method for dynamic eventcapturing and rendering of claim 17, wherein the one or more processorscorrelate the location of the object with an overhead video rendering ofthe field of play.
 27. The method for dynamic event capturing andrendering of claim 17, wherein the one or more processors display on auser interface a select portion of the 360-degree rendering of a firstobject along with the location of the first object on the field of play.28. The method for dynamic event capturing and rendering of claim 27,wherein the one or more processors display on a user interface a selectportion of the 360-degree rendering of the one or more other objectsalong with the location of the one or more other objects on the field ofplay, the location of the first object on the field of play and theselect portion of the 360-degree rendering of the first object.
 29. Themethod for dynamic event capturing and rendering of claim 17, whereinthe one or more processors integrate the 360-degree rendering of theobject along with the location of the object on the field of play with avirtual reality environment related to the field or play.